Experiences with building an intrusion-tolerant group communication system

There are many group communication systems (GCSs) that provide consistent group membership and reliable, ordered multicast properties in the presence of crash faults. However, relatively few GCS implementations are able to provide those properties in the presence of malicious faults resulting from intrusions. We describe the systematic transformation of a crash-tolerant GCS, namely C-Ensemble, into an intrusion-tolerant GCS, the ITUA GCS. To do the transformation, we devised intrusion-tolerant versions of key group communication protocols. We then inserted implementations of the protocols into C-Ensemble and made significant changes to the rest of the C-Ensemble protocol stack to make the stack intrusion-tolerant. We quantify the cost of providing intrusiontolerant group communication in two ways. First, we quantify the implementation effort by presenting a detailed analysis of the amount of change required to the original C-Ensemble system. In doing so, we provide insight into the choice of building an intrusion-tolerant GCS from scratch versus building one by leveraging a crashtolerant implementation. Second, we quantify the run-time performance cost of tolerating intrusions by presenting results from an experimental evaluation of the main intrusiontolerant microprotocols. The results are analyzed to identify the parts that contribute the most overhead while providing intrusion tolerance during both normal operation and recovery from intrusions.